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Abstract 

In this thesis work we are concerned with development of a content based indexing 
and retrieval system This is based on -wavelet decomposition and quad tree segmen- 
tation Since the computational complexity has been one of the mam barriers towards 
the use of similarity measures in large image databases we propose a hierarchical in- 
dexing scheme where computationally efficient features are used to subset the images 
before more sophisticated techniques are applied for precise retrieval Database sys 
terns which support indexing searching and retrieval have great demand Large visual 
database systems require effective and efficient ways of indexing and accessing visual 
data on the basis of content In our work we use 3 level wavelet transform to extract 
image features, significant features must first be extracted from image data in their 
pixel format Feature vectors of images are then constructed These feature vectors 
of database image segments are classified using clustering algorithm Hierarchical 
tree structure namely Bucket PR k d tree is used for efficient storage and searching 
of the clusters Content based image retrieval is performed bj comparing the feature 
vectors of the query image and the mean feature vector of clusters With the large 
volume of visual data stored in a visual database image classification is a critical 
step to achieve efficient indexing and retrieval Our experiment illustrates that the 
proposed block oriented image representation offers a novel decomposition structure 
to be used to facilitate effective and efficient image retrieval 


Chapter 1 


Introduction 


Digital images have become the medium of choice for describing storing and ex 
changing information in a variety of domains e g satellite images, biomedical images 
stock photo archives of news agencies etc With the decreasing cost and increasing 
performance of digital image/ video capture devices, computing power and storage 
capabilities, more and more visual information are now available on line m large 
image and video repositories However image and video data are much more volu 
minous than textual data The amount of data associated with visual information 
requires enormous storage capacity As a result image data compression techniques 
are used for reduction of the number of bits required to store or transmit images 
without an} appreciable loss of information Among the image compression methods 
JPEG wavelet transformation and frarteJ coding [4] have recently drawn much at ten 
tion Recent developments in the field of image data transformation and compression 
provide an interesting approach to describe the contents of an image [8] 

Content based retrieval of imagery [3] [8] [9], [10] is presently an actne area 
of research promising to provide powerful tools for database management in the 
near future TYaditional methods of retrieving images from such a database rely 
on a number of descriptive keywords associated with each image These methods 
are inadequate m terms of effective description of the image contents because they 
are subjective(relative to the person indexing the images) Thus modern database 
systems should process images based on their content 

Content based image retrieval has been proposed to allow retrievals to be per 
formed on the basis of different aspects of the image content In this context a 
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challenging problem arises with mam image databases in which queries are posed 
via \isual or pictorial examples Such queries are termed as visual queries A com 
mon visual query to a image database sjstem would involve finding all the images 
in that database which contain a subimage that is similar to a given querv image 
Such retrievals must use embedded content features, such as the shape color, texture, 
layout and the positions of various objects in an image There are no generic tools 
which helps to understand the image content to a satisfiable extent The automatic 
retrieval of images on the basis of content thus pose difficult problems An approach 
which has drawn much attention recently involves the extraction of the color and 
texture features of images using image processing techniques and form image feature 
vectors based on the color and texture characteristics Content based image retrieval 
IS then supported by searching and comparing the feature vectors of the query im 
age with the database images Database systems which support indexing searching 
and retrieval of image are in great demand Different approaches adopted for these 
databases include 

1 Multiresolution image representations for browsing and retrievals at various scales, 

2 Textual description of data for keyword indexing, 

3 Feature sets for searching by content e g , texture, color, shape spatial relation 
ships, etc 

Multiresolution(MR) image representations allow the user to browse through databases 
b> view mg thumbnail sketches of image data But even with multiresolution brow sing 
and retrle^al, it is difficult to search through large amounts of data So content based 
approaches have been adopted which allow searching based on visual features of the 
image data In this process significant features are extracted from image data in 
their pixel format 

There are many steps involved in the implementation of a feature-based retrieval sys 
tern They are 

1 Feature set identification, 

2 Feature space compaction 

3 Image segmentation and 

4 Multidimensional point indexing 

The important visual features of an image are described mathematicallj using 
feature sets that are derived from the digital data A content based search of the 
database proceeds by finding the items that are mathematically and visualh similar 
to the query image Characterization of texture color and shape using feature sets 
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are being used m content based approaches for large image and video databases 
Classification and discrimination [10] [4] of textures can be done based on 

the energies of image subbands Subband energy feature sets can be extracted from 
several typical image decompositions such as wavelet subband uniform subband, 
discrete cosine transform (DCT) and spatial partitioning In our present work we 
use Wavelet transforms for feature extraction Feature vectors of images are then 
constructed The subband energj based feature sets are applied to a sjstem for 
indexing images by texture content in image databases, since the features can be 
extracted directly from spatial frequency decomposition of the image data These 
features are classified and indexed to assist efficient retrieval of image content For 
the efficient representation of database images we use an effective block oriented 
image decomposition structure yvhich can represent image content in image database 
systems Content based image retrieval is performed by comparing the feature vector 
of the query image and the decomposed segments in database images 

The rest of the report is organized as follows Chapter 2 describes image data 
segmentation and introduces the tree block oriented data structure namelj quad tree 
In chapter 3 we discuss texture-based query and wavelet transform for image data 
compression and feature extraction This is followed by a description of a procedure 
to find the textural similarity between database images and the querv image In 
chapter 4 we discuss the criteria used for grouping or clustering data followed by a 
procedure to store the clusters in a hierarchical tree structure namely bucket PR k d 
tree for easy and quick retrieval of database images for a given query Experimental 
results presented in chapter 5 Chapter 6 concludes the thesis and suggests the scope 
for future work 



Chapter 2 


Image Segmentation 


Image segmentation refers to the decomposition of a scene into its components It is 
a key step m image analysis For example a document reader would first segment the 
various characters before proceeding to identify them The Figure belov lists several 
image segmentation techniques 

Image Segmentation Techniques 


Amplitude Component Boundary based Region Based Template Texture 

Thresholding labeling approaches approaches matching SegmentaUo 

and clustering 

Figure 2 1 Different Segmentation Techniques 

Amplitude thresholding is useful whenever the amplitude features sufficiently char 
actenze the object The appropriate amplitude feature values are calibrated so that 
a given amplitude interval represents a unique object characteristic For example 
the large amplitudes of the remotelj sensed visual and infrared (IR) images can rep- 
resent low temperatures or high altitudes etc Component labeling is a simple and 
effective method of segmentation of binary images by examining the connectivity of 
pixels with their neighbors and labeling the connected sets Two practical algorithms 
used for connectivity analysis are Pixel labeling and Run length Boundary extrac 
tion techniques segment objects on the basis of their profiles Edge linking curve 
fitting etc are also applicable to image segmentation Template matching is a direct 
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method of segmenting an image in which an image is matched against templates from 
a given list The detected objects can be segmented out and the remaining image can 
be analyzed by other techniques This method can be used to segment busy images, 
such as journal pages containing text and graphics Texture segmentation becomes 
important when objects m a scene have a textured background In our work, we have 
used the region based approach to segmentation 


2 1 Region-Based Approaches 

As our mam approach is to adapt the region based techniques we discuss this tech 
niques here in detail The mam idea of region-based segmentation techniques is to 
identify various regions in an image that have similar features Clustering techniques 
encountered m pattern recognition literature have similar objectives and can be ap- 
plied to image segmentation 

One class of region-based techniques involves region growing The image is divided 
into automatic regions of constant gray levels Similar adjacent regions are merged 
sequentially until the adjacent regions become sufficiently different 

Instead of merging regions segmentation is done b\ splitting a given region 
For example , the image is split by Quad-Tree approach and then similar regions 
are merged Region based approaches are generally less sensitive to noise than the 
boundary-based methods In the next section we discuss the Quad tree decomposi 
tion 

2 2 Quad- Tree Decomposition 

As the feature vector of a database image may not correctly represent its subimsges 
the retrieval based on the comparison between the feature vectors of the query im- 
age and database images themselves may not provide satisfactory results for visual 
queries Thus image segmentation is a necessary step for effective searching of image 
databases Effectne segmentation will isolate the important homogeneous regions 
and features of the images m the database, from which an index can be established 
for searching To avoid manual segmentation of images in a large database,a block 
oriented approach based on the quad-tree decomposition of images has been adopted 

A quad tree [1] [2] [3] is a hierarchical image decomposition structure which can 
provide quick data access for image retrieval as hierarchical data structures have the 
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ability to focus on the interesting subsets of the data This focusing results in an 
efficient representation and improved execution time A. quad tree is based on the 
principle of recursne decomposition of space The most studied quad tree approach 
to region representation termed as a region quad tree is based on the successive 
subdivision of the image array 


root 



(a) 


NW 

NE 

SW 

SE 


(b) 


Figure 2 2 Quad Tree decomposition of an image arra\ a) Tree representation of 
the image arra\ b) Decomposed image array 


Each decomposition of an image segment produces four equal sized quadrants 
Figure 2 2 demonstrates the positions of the four quadrants labeled m order NW, NE 
SW and SE iMthin the decomposed segments The leaf nodes of the tree correspond 
to those blocks for which no further subdivision is necessary All non leaf nodes are 
said to be GRAA The tree decomposition and block decomposition are shown m 
figure 2 2 

The image decomposition process using a quad tree structure can be described 
recursively, with the root representing the entire image and its children represent 
mg the decomposed segments, these, in turn become roots for further segmental 
decomposition Each internal node has exactly four children The original quad tree 
decomposition labels the decomposed segments white if they consist of white pixels 
only, black, if the^ consist of black pixels only and graj, if the> consists of both black 
and white pixels Further decompositions is carried out onlj on gray segments Black 
and white segments remain unchanged 

We have done the transformation and decomposition of an image array as follows, 
firstly, the entire image is decomposed using wavelet filtering then quad tree spatial 
blocks are obtained to represent appropriate regions m the original image array Then 
for these blocks we take wavelet transformation to obtain the feature vector of each 
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NW 
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(a) (b) (c) 

Figure 2 3 Splitting examples (a) NO Split all children belong to single class (b) 
Split into two children NW and SW form a texture class and children NE and SE 
form a second texture class (c) Split into four four texture classes present m this 
spatial block 


NE 


NW 


SW 


SE 


NE 


NW 


SW 


SE 


block In this way, at each level we get the feature vector for each segment of the 
image arraj But the procedure we followed to proceed for next level segmentation is 
shghth different We examine at each level the merging condition of the blocks using 
a distance threshold that is discussed in the next chapter 

Now we discuss the procedure that we follow to combine the image data compres 
Sion or transformation and the image data segmentation Here we have implemented 
a changed definition of leaf nodes Before the four children are spawned bj each 
parent condition of merging are tested The distance threshold is computed for each 
child on the basis of the extracted texture features The Euclidean distances in the 
feature space are measured from the parent node to each child If the distances to 
all four children fall within the threshold of the corresponding child, a single tex 
ture would be declared m the parent node and no further decomposition is necessary 
Otherwise pairwise grouping of the children are performed That is, if the Euclidean 
distance between two neighboring children falls below their respective thresholds, the 
children are merged as a single child Different situations may occur depending on the 
distance between nodes This is shown m figure 2 3 The quad tree decomposition 
is then iterated on each child So the quad tree is grown iteratively by testing the 
condition for splitting a parent block based on the texture content of children The 
quad tree decomposition can be characterized as a variable resolution data structure 
An example of a quad tree decomposition is gi\en below in figure 2 4 
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Figure 2 4 a) Original image b) Quad tree representation of the image 












Chapter 3 


Feature Extraction in 
Texture- based Image Query 


Gi\en the large volume of image data collected m a visual database, a manual ap- 
proach for detecting and classifying images is both highly ineflBcient and prone to 
errors arising from subjective interpretation An approach which has drawn much 
attention recently involves the extraction of the texture features of images from their 
mathematical representations used in various image compression techniques 

The visual characteristics of homogeneous regions of real world images are often 
identified as texture These regions may contain unique visual patterns or spatial 
arrangements of pixels which regional gray-level or color alone mav not sufficiently 
describe Typically, textures have been found to have statistical properties struc 
tural properties or both Texture is an important underlying primitive in human 
visual perception including coarseness contrast and directionalitj These texture 
characteristics can be used to support content based image retrieval 

Texture is used to describe content of many real world images for example, 
clouds, trees bricks, hair,fabric all have textural characteristics Because of the fun- 
damental importance of texture information for human vision texture can provide 
a meaningful tool for searching image databases By indexing on the texture con- 
tents of images in the database a user may search through large volumes of images 
using texture keys Ideallj there is no restriction that the texture belongs to pre- 
defined classes Furthermore the Query by texture” can be combined with other 
descriptions of color and shape to formulate overall image content based queries 
Towards the development of Query by texture” , it is necessary to find a mean 
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ingful measure of the similant\ of textures (texture discrimination) and to develop 
procedure for segmenting images based on textual content(te\ture segmentation) 
Furthermore discriminant functions are needed to gauge the similaxitv between the 
texture regions and the texture key used for searching Likewise the texture discnm 
inant functions can be used to match blocks within each image to produce the texture 
segmentation The goal of this segmentation is to pro\ ide indexing of images in the 
database Segmentation is successful when important regions of texture of the images 
are obtained Using the spatial quad tree approach each quad tree node points to 
a block of image data Children nodes are merged when the discriminant functions 
indicate that the children blocks contain sufficiently similar textures A ’ Query-by 
texture’ examines the blocks identified by the final quad tree structures to test the 
similarity to a texture ke> used for the search 


3 1 Textural Similarity 

The Brodatz texture collection (Brodatz images) was used to obtain discriminant 
functions (equation 3 1) This procedure, used in our work, constructs linear com 
posites of the features which provide maximum average separation among training 
classes The Mahalanobis distance [15] which is the squared distance in the trans 
formed feature space was used to measure the similarit> between textures This is 
given by equation 3 2 In ordinary classification of textures or comparisons of many 
textures the relative rankings of the Mahalanobis distances are used to identify clos- 
est matches Subsequentlj m response to a ’ Query b\ texture all textures in the 
database will be sorted b\ this distance to the texture-ke> This search needs no 
threshold to be established to determine when textures are no longer "similar ’ How 
ever, to decide whether two textures are similar or not as in quad tree segmentation 
requires a threshold in distance 

Using a fixed distance threshold for determining whether two textures are e suf 
ficiently similar will not be optimal for all types of textures Depending on the 
characteristics captured by the extracted feature sets the within class variance for 
visibly similar textures will vary from class to class The distance threshold depends 
heavily on the block size from which the features are extracted and on the energy of 
the feature set 

There is a relationship between a block size and the estimated feature probability 
distribution of the block Since smaller texture blocks will contain fewer data points 
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from which to den\e statistical features there will be a larger deviation m features 
extracted from these blocks of similar textures This results in a greater variance m 
distance between possibly similar textures necessitating a higher distance threshold 
for comparing textures of smaller block size Likewise features extracted from larger 
blocks produce smaller within class variation necessitating a lower distance threshold 
for comparing textures of larger block size 

The distance threshold can be determined from information about the textures 
that are compared namely image region size and energy of the feature set Performing 
a linear regression analysis on this data, the threshold function is given as 

t/i = 82 057* II /u||/s- 1 748 (3 1) 

Here th is the threshold to be used m the Mahalanobis distance, ||/u|| is the energy 
of the transformed feature vector and s is the number of pixels in the image block 


3 2 Searching Quad- Trees 

After decomposing the image array m a quad-tree we search through the tree to find 
out the matching between the query image and database image Each node of the 
tree contains the feature vector of a segment rather than the original pixel data Also 
the feature vector is generated only on the basis of the pixel data rather than the 
entire image 

A relationship between a given query image and am image m the database is 
determined bj comparing the feature vector of query image with different portions 
1 e different nodes of the database image until a match is found This is done by 
traveling through the quad tree of the database image from the root to the bottom 
We use the root mean square metric to compare the distance between the feature 
vector of the query image and that of the image segments of the database images 
For the given feature vectors ul(ai,a 2 , ,a„) and V 2 (bi ,b„), of database image 
and query image respectively, their distance is computed as follows 

dtsi(vl,v2) = \f{l/n - b^f) (3 2) 

I 

For a given threshold if the distance between the quen image and a segment of 
a database image is less than the threshold, then we select this database image as a 
matched image 


! 1 r r ' 
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But this direct search is not efficient as using this we have to search a large number 
of nodes So in next chapter we discuss the clustering or grouping of feature \ectors 
of the nodes of the quad tree representation of the database image 


3 3 Wavelet Transforms 

In computer vision it is difficult to analyze the information content of an image di 
recth from the gra> level intensity of the image pixels Indeed, this value depends 
upon the lighting conditions Generallj the structures to be recognized ha\e very 
different sizes Heni.e, it is not possible to define a prion an optimal resolution for 
analyzing images To process the image at different resolution, image information 
can be reorganized into a set of details appearing at different resolutions A mul 
tiresolution decomposition enables us to have a scale invariant interpretation of the 
image A multiresolution representation [11] provides a simple hierarchical frame 
work for interpreting the image information At different resolutions the details of 
an image generally characterize different physical structures of the scene At a coarse 
resolution, these details correspond to the larger structure which provide the image 
’’context” It IS therefore natural to analyze first the image details at a coarse resolu 
tion and then gradually increase the resolution Such coarse-to-fine strategy is useful 
for low level image processing such as stereo matching and template matchmg 

Multiresolution repi esentations are yery effective for analyzing the information 
content of images the different of information between the approximation of the 
signal at the successive resolutions can be extracted by decomposing this signal on a 
wayelet orthogonal basis This decomposition defines an orthogonal multiresolution 
representation called a wavelet representation It is computed with a pyramidal al 
gorithm based on the convolutions with quadrature mirror filters and this is why this 
computation is efficient 

Like the first Fourier transform(FFT) the discrete wavelet(DWT) is a fast linear 
operation that operates on a data vector whose length is a power of two transforming 
it into a numerically different vector of the same length Also like the FFT, the wavelet 
transform is invertible and m fact orthogonal the inverse transform, when yiewed 
as a big matrix, is simply the transpose of the transform So both FFT and DWT, 
therefore can be viewed as a rotation in a function space, from the input space (or 
time) domain to a different domain 

There are many possible sets of wavelets A particular set of wavelets is specified 
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by a particular set of numbers called wavelet filter coefficients Here a class of wavelet 
filters discovered b\ Daubechies are used by us This class includes members ranging 
from highly located to highly smooth The simplest and most localized member often 
called DAUB4 has only four coefficients, cO cl c2 c3 

The DWT consists of applying a wavelet coefficient matrix hierarchically, first to 
the full data vector of length N then to the smooth vector of length JV/2 then 
to the smooth smooth” vector of length N/4 and so on until only a trivial number 
of Smooth Smooth components remain This procedure is called pyramidal algo- 
rithm The output of the DWT consists of these remaining components and all the 
’’detail components that were accumulated along the way 

A wavelet transform of a 2-dimensional array is easily obtained by transforming 
the array sequentially on its first mdex(for all values of its other indices) then on its 
second and so on Each transformation corresponds to multiplication by an orthogonal 
matrix 


columns 



Figure 3 1 Decomposition of an image into and D^J This 

algorithm is based on the one dimensional convolution of rows and columns of A^^^J 
with the one-dimensional quadrature-mirror filter G and H 


An immediate application of the multidimensional wavelet transform is in image 
data compression For images the wavelet representation differentiates several spatial 
orientations The two-dimensional wavelet transform can be seen as one dimensional 
wavelet transform along the x and y axes Wavelet based subband coding allows 
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simultaneously for high spatial resolution at high spatial frequencies and high spatial 
frequency resolution at lo^v spatial frequencies Thus a filter bank based on wavelets 
could be used to decompose the image into low pass and high pass spatial frequency 
bands In two dimensional wavelet transform at each step we decompose into 

^ijf ^Ijfy ^ 2 jf ^nd D\J This algorithm is illustrated bv a block diagram m 
Figure 3 1 We first con\olve the rows of A f with one dimensional filter retain 
every other row convohe the columns of the resulting signeds with another one 
dimensional filter and retain every other column The filter used in this decomposition 
are the quadrature mirror filter (QMF) Wavelet transforms are closely related to tree 
structured filter banks cind hence to multiresolution analysis Tree structured fiilter 
banks give rise to nonuniform filter bandwidths and nonuniform decimation ratios 
in the subbands These two non uniformities are the fundamental ingredients of the 
wavelet transform Wa^eIet based subband coding allows simultaneously for high 
spatial resolution at high spatial frequencies and high spatial frequency resolution 
at low spatial frequencies Thus a filter-bank based on waielets could be used to 
decompose the image into low pass and high pass spatial frequencv bands 

Orthogonal wavelet transforms provide interesting insights on the statistical prop- 
erties of images Each of the subbands obtained after filtering have uniform texture 
information Feature extraction can be performed by energy estimation in subbands 
The feature used are computed from the mean absolute value and variance measures 
on the subbands produced from three iterations of QMF wavelet decomposition The 
advantage of these wavelet spatial frequency approaches is that simple statistics com 
puted from the subband images are used as the subbands have limited spatial mfor 
mation Wavelet representations also provide a multire&olution structure and energy 
compaction to enable compression 

We have used wavelet transformation in our work The wavelet decomposition 
and feature extraction processes are no longer independent for each spatial block At 
first, the entire image is decomposed using wavelet filtering then quad tree spatial 
blocks point to appropriate regions in the full original image Wavelet transform is 
applied to these regions to extract the feature information and construct the feature 
vectors for respective blocks This is shown in figure 3 2 
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Figure 3 2 Quad tree blocks points to the appropriate regions in the full original 
image for feature extraction 






Data Clustering 


17 


example data maj crow d into several partitions while other remain sparse or empty 
This result is nonuniform search performance 

4 12 Data Clustering 

An improvement over partitioning the feature space is to gather data points into clus 
ters In unsupervised learning we attempt to identify clusters or natural groupings 
in the feature space A cluster is a set of points in the feature space with local den 
sity that IS large (relative maximum) compared to the density of the feature points 
in the surrounding region As we have feature space which contains vectors whose 
components are real numbers, a feature class can be characten 2 ed by its clustenng 
properties m the feature space Clustering techniques Me useful for image segmenta 
tion and for classification of row data to establish classes and prototypes Clustering 
IS also a useful vector quantization technique for compression of images 

The feature space search can be broken into two stages The first will identify 
the appropriate cluster and the second will search all the data points m that cluster 
Search performances at each stage can be regulated by the clustering algorithm For 
example, a bound set on the maximum number of data points per cluster will produce 
a bound on the second stage of the search algorithm This results in more consistent 
performance than partitioning the feature space 

The success of clustenng techniques rests on the partitioning of the feature space 
into cluster subsets A general clustering algorithm is based on split and merge ideas 
shown in figure 4 1 below 



Figure 4 1 A clustenng approach 


If the vectors are characterized by clusters which are far apart simple recognition 
schemes such as minimum distance classifiers where the features are classified by 
distance functions, may be successfully employed The motn-ation for using distance 
functions as classification tool follows naturally from the fact that the most obvious 
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way of establishing a measure of similarity between feature vectors which we consider 
as points in Euclidean space is bv determining their proximity 

The method of feature classification by distance functions can be expected to 
yield practical and satisfactory results only when the feature groups or clusters do not 
overlap When the clusters overlap it becomes necessary to utilize more sophisticated 
techniques for grouping the feature space Overlapping clusters are the result of a 
deficiency in observed information and the presence of measurement noise Hence the 
degree of overlapping can often be minimized by increasing the number and quality 
of measurements performed on the feature vectors 

As the database consists of a number of images so searching of individual qucid 
tree which represents a database image may be highly time consuming This problem 
IS circumvented by a novel clustering and indexing technique on the feature vectors 
generated from the image segments Before indexing the database images features 
vectors are grouped depending on diflferent criterion 

Data grouping is not a rigorously or uniformly defined concept Some speak of it 
as the subdivision of a set of data into subsets whose members are similar m some 
sense Others seek subsets exhibiting a cohesiveness which is not entirely related 
to point to-pomt similarity In the most general case, the proper number of subsets 
is unknown Here we consider a case where the number of subsets is fixed Many 
procedure for solving the general problem combine an algorithm for grouping data 
into a fixed number of subsets with some procedures for merging or splitting subsets 
Two basic techniques for grouping data appear in the literature 

1 Hierarchical approach 

2 Direct approach 

In hierarchical approach, data is divided into subsets and each subset is divided into 
further subsets and so on until the desired number of subsets is created The direct 
approach, on the other hand is to subdivide the data into the desired number of 
subsets m one step The direct approach seems to yield more reasonable classification 
but require more computation Hierarchical classification may require only the ability 
to divide data into two subsets 

Data grouping is often accomplished with the aid of a criterion This criterion 
assigns a number to each possible partition of the data The partition which yields an 
extreme value of the criterion is chosen as the desired partition The criterion must 
meet tw o basic requirements 

Performance The resulting partition must fall along natural boundaries of the data 
when such boundaries are well defined 



Criteria and algorithms for Grouping Data 


19 


Efficiency There must exist an efficient algorithm for finding optimum partition 
Another requirement is uniqueness We would like to ha\e one partition which 
extremizes our criterion Although uniqueness is an important factor, we consider 
here only performance and efficiency 

We discuss in this chapter the criterion to be considered for grouping a basic 
algorithm for clustering the data and then define a linear transformation [13] which 
can be applied to the original data to improve the performance of the clustering 
algorithm 


4 2 Criteria and Algorithms for Grouping Data 

The criterion which we consider here can be defined in terms of the statistical scatter 
matrices introduced b^ Wilks [19] Let Xi X 2 , ,X„ a set of L-dimensional column 

vectors, be the data set We wish to divide this set into M groups Gi G 2 , Gm 
with populations A 2 respectively The scatter matrices are defined as 

follows 


Total scatter 

T,^-£X,Xl 

(4 1) 

Intragroup scatter 

k-l 

t £ (X* - C,)(X* - C,Y 

(4 2) 

where 

Xk^C, 

= E 

■' V Xk^Gj 

A ^ 

(4 3) 

Total intragroup scatter 

(4 4) 

Intergroup scatter 

b = '£n,c,cJ 

1 = 1 

(4 5) 

The superscript T denotes transposition It is easily showm that 



Tl = W + B 

(4 6) 


regardless of the partition Another important propert\ of the scatter matrices 
IS the fact that the eigemalues of W~^B denoted by Aj A^, are invariant under 
nonsingular linear transformations of Xk s 
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Criteria for Grouping 

Several criteria ma> be defined m terms of scatter matrices definedl above The 
intragroup scatter is a measure of the degree of association between the members of 
the group The criteria we consider here is based on some measure of the magnitude 
of W, the intragroup scatter 
The criterion Jq is defined as 

M 

Jo=trW = '£ 

j=i Xi^eGj 

where trW is the trace of the matrix W Jq may be interpreted as the mean squared 
distance to the group center The optimum partition is taken as the one which 
minimizes Jq 

Jq is not imariant under nonsingular linear transformations of the Xk s This 
means that by changing the coordinate system of the original data, one may alter the 
optimum position A criterion of the form 

J=/(Ai ,Al) (4 8) 

is invariant under nonsingular linear transformations Here the optimum partition is 
the one which maximizes the criterion 

A basic assumption m clustering is that the group structure is somehow reflected 
by the data If we feel that this structure is reflected in an invariant manner, then 
we should use an invariant criterion to detect it This was essentially what Friedman 
and Rubin [20] assumed and the notion was supported b> their experiments 

4 3 Algorithm 

The basic algonthm shown in Fig 4 2 has been widel> used and in our case, we have 
used this algorithm This algorithm yields a grouping of the data which minimizes, 
at least locall} Jo 

Jq is good in terms of the efficiency requirement If the coordinate system is 
suitably chosen Jo becomes a function of Aj, ,Al, thereby becoming invariant to 
any nonsmgular linear transformation on the vector space Prom our definitions it 
follows that if A > L the total scatter matrix Tl is a positive definite matrix Then, 
there exists a nonsmgular matrix A, such that 

ATlA^ = / (4 9 ) 
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Figure 4 2 Basic grouping algorithm 


regardless of the partition In fact as Tl is positive definite the matrix A is actu 
ally the eigen vector matrix of the total scatter matrix Ti The nonsingular linear 
transformation A diagonalises the total scatter matrix and makes Jo a function of 
Ai, , Ajr,, which takes the form 

(410) 

thereby making the criteria invariant to any nonsingular linear transformation The 
application of this nonsingular linear transformation to the data set before actually 
starting the basic grouping algorithm improves the performance of the clustering 
algorithm 
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4 4 Formation of Hierarchical Data Structure for 
Cluster Storage and Retrieval 

Using the clustering algorithm mentioned above, we get two clusters at each stage As 
there is an upper bound on the maximum number of data points per cluster, the pair 
of clusters formed at each stage may require further sub-clustering depending on the 
number of data points in that particular cluster All these clusters and sub-clusters 
need to be represented in the form of an appropriate hierarchical data structure for 
proper storage and later on for quick and easy retnevcils of the matches to a given 
query image For this purpose, we have used a spatial data structure similar to a 
Bucket PR k d tree [21] as shown in Figure 4 3 A k d tree typically handles a 
point query determining if a given data point is in the database and if so yields the 
information corresponding to the record m which it is stored In the term k d tree 
k denotes the dimensionality of the space being represented Typically, it is a binary 
search tree with the distinction that at each depth a different attribute or key value 
is tested when determining the direction m which a branch is to made 

k-d trees are useful in applications involving search Let us consider a typical 
query that seeks all nodes within a specified distance of a given point The k d tree 
structure serves as a pruning device on the amount of search required, i e many 
nodes need not be examined Assuming a k-dimensional space at each node, only 
one key needs to be compared instead of k In our case this key is the Euclidean 
distance between the query data point and the mean of the feature vectors of the two 
clusters Terminal nodes, as well as other nodes do not occupj too much space since 
the number of pointers per node is reduced to 2 for all values of k instead of 2* 
Apart from the point based trees, eg k d tree there are a number of ways of 
adapting the region based trees, like MX quad trees [21] PR quad trees [21], to rep- 
resent point data The MX quad tree is feasible as long as the domain of data points 
IS discrete and finite If this is not the case, the data pomts cannot be represented 
using this because the minimum separation between data points is unknown This 
observation leads to an alternative adaptation of the region quad tree to point data 
which associates data points(which need not be discrete) with quadrants typically 
We call this PR quad free(where P stands for point and R stands for region) 

In our work, we describe k d tree as an analogous data structure that uses binary 
trees instead of quad trees Such a data structure can also be termed as a PR k d 
tree or a PR bin tree 
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Figure 4 3 PR k d tree and the records it represents 


Using standard tree structures for unage databases is a bit impractical smce a tree 
structure requires that pointers be followed and a point query results m the retneval 
of the closest matching pomt m the database If we have similar image segments 
present in different parts of the same image or in multiple images present m the same 
database, and if we give a sumlar query image, we would like to retrieve all matching 
images from the database This is not possible in a PR k-d tree smce each quadrant 
or cluster can contain at most one data point To overcome this problem, methods 
have been designed to coUect the pomts mto sets(termed Buckets) Then the access 
of these buckets is organized by the use of an appropriate tree to speed up the process 
of image retneval We term such techniques as bucket methods” Having buckets 
of capacity c(c > 1) reduces the maximum dependence of the PR k-d tree on the 
minimum Euclidean distance separation of two distinct points to that of two sets of c 

points The term Bucket PR kd tree IS used to refer to this structure [21] In our case 

unlike bucket PR k-d tree which divides the data set into quadrants m k-dimensional 
vector space, we form two sub-clusters at any stage, m the k dimensional vector space, 
from the existing points m the cluster under consideration This is carried out ^ long 
as It overflows i e the number of data points in the cluster is greater than the hu e 
size The process, of sub-cluster formation continues until all the tennmal nodes 
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Figure 4 4 Two clusters and sub-clusters with theu: tree representation 


have partially filled bucket or at most full buckets In Figure 4 4 we have shown an 
example of the clustermg process using the hierarchical tree structure i e bucket PR 
k d tree First the feature space is divided into two clusters, then within each cluster 
sub-clustenng is done Each group contains data points that is less than or equal to 
the bucket size The tree representation of this clustering process 

For a given query image, retneval of relevant mformation is done m a standard 
way as in the PR k d tree In the present work, the only difierence is that , the 
query causes the retrieval of all the data pomts m the bucket corresponding to the 
terminal node to which the query data pomts has been gmded to and present them 
in a suitable manner dependmg on their proximity to the query data pomts 



Chapter 4 


Indexing The Database Images 


4 1 Hierarchical Searching 

As the number of images and texture regions increase, it becomes necessary to utilize 
a seaichmg procedure through the database more efficient than exhaustive search 
The ordering of the discriminant functions by significance is a natural bj product 
of the feature space compaction performed by the Fisher Discriminant Analysis [10] 
The energy compaction may be exploited by adapting a hierarchical approach to 
searching Therefore, each dimension of the feature space is searched independently 
in order of significance For example a binary search may be conducted on sorted 
feature data beginning with the most significant discriminant functions At first, the 
candidate match list may be kept larger than the final desired number of retrievals As 
the list IS subsequently searched using the remaining discriminant fuii.tions it may 
be further truncated This technique of successive refinement reduces the complexity 
of searching the high dimensional feature space by reducing the oveiall number of 
comparisons 

4 11 Feature Space Partitioning 

A second alternative to searching the high dimensional texture feature space is to 
partition the feature space This wa\ the search can be conducted in tViO stages 
The first will identify the appropriate partition in feature space The next stage 
will examine only the data points in the partition However, if the dimension of the 
feature space is large, it is unlikeh that uniform partitioning will be optimal For 


Chapter 5 


Experiments 


A testbed of the database has been constructed from 15 Brodatz texture images 
The test images are 128*128 pixels m size The quer> images were chosen as random 
subimages of randomh chosen Brodatz images of size 32*32 pixels 

Database images were decomposed using the quad tree decomposition procedure 
as explained in chapter 2 Feature extraction was performed on all image segments 
m these trees using wa\elet transform which filter out texture features of images 



Input Image Wavelet Image Feature Extracuon 


Figure 5 1 Tevture classifier for Brodatz textures using subband energy based fea 
tures 


Three l(\ds of subband decomposition were used m our experiments Each of 
the subbands obtained after filtering have unifoim texture information In our ex 
penment featuie exti iction was performed using Daubechies 4 wavelet filter Each 
three le\el transformation produces 10 subbands or image blocks as shown in Figure 
3 1m chapter 3 Features were extracted bv energe estimation in subbands Two 
energj features mean and variance were computed for each subband i e for each 
decomposition energns were measured by calcul itiug the variance and mean abso- 
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lute \alue of each subband Thus each feature \ector has 20 elements This feature 
extraction procedure was applied to all image segments in the tree representation of 
each image Each node in the tree thus represents a segment of the image and its 
feature \ector The feature space contains all generated feature xectors of image seg 
ments \ block diagram representation of the feature extraction process is presented 
in Figure 5 1 

Database image segments were then grouped using the clustering algorithm dis 
cussed in chapter 4 (Figure 4 4) To get proper clusters we used a linear transforma 
tion on feature vectors in the feature space [21] Then we constructed a hierarchical 
tree structure named Bucket PR k d tree (Figure 4 3) for cluster storage and eas\ 
and quick retrieval The feature vector for each que^^ image was calculated using the 
root mean square( rms) metric This was used to compare the distance between the 
feature \ ectors of the query image and the mean feature vectors of each node of the 
bucket PR k d tree At the end of the search procedure when a bucket is reached the 
database image segments which fall in that particular bucket are retrieved as matched 
image segments In this way we get the information about the database images which 
contain a segment /segments which is/are similar to the query image The flow chart 
gi\en in Figure 5 1 explains the steps we followed m our experiment 

Database images belonging to different texture groups but containing closelj sim 
liar texture features may fall within a verj small distance Also some semanticalh 
irrele\ant images may have feature \ectors that fall within a very small distance 
To pre\ent the retrieval of such lrrele^ant images clustering approach is needed to 
classify the images into different categories befoie rctnexal is performed 


5 1 Result 

Results of the query are shown for two quer\ images 

Figure 1 of the result shows four matched database images Each of them contains 
a segment that is similar to the quer} image Table 1 shows the matched image 
segments for this query The second column sho^\ s the name of the matched database 
images and the third column shows the ttus distances between the query image and 
the corresponding database images Database images or image segments belonging 
to different texture groups but containing closeU similar texture features ma\ fall 
within a very small distance As indicated in this Table some irrelevant segments 
were retrieved as matched segments of th( database images This means that though 
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the\ belong to two different texture groups m Brodatz texture images that were 
used to construct our database the texture features in the different segments contain 
closeh similar texture features The retne\al result also depends on the bucket size 
If the bucket size is too small some rele\ant information might not be retrie\ed On 
the other hand, if the size is too big there is a possibilit\ of getting some irrele\ant 
information So we kept the bucket size to an optimum \alue 10 Referring to the 
distance column in Table 1 we observe that distances for the irrelevant segments are 
higher than that of relevant segments Image segments are sorted here according to 
their distances with the query image 

Figure 2 of the result shows four relevant images Table 2 shows onlj the relevant 
segments of these images 




Figure 1 Name of the matched database images in order db9 pgm 
dbl pgm dblO pgm db7 pgm 
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Figure 5 2 I lowchart of the content based indexing and retries svstem 










Chapter 6 


Conclusion 


In this thesis we have presented a method for performing Query b\ texture on an 
image database The proposed database is tightly coupled with an image compres 
Sion algorithm using which feature vectors are constructed and used for clustering and 
quick searching of the database Texture has been shown to be a visible characteristic 
of visual information by which image data may be indexed in a large image database 
The method employed uses a quad tree approach to image segmentation and applies 
wavelet transformation to extract feature sets from image blocks and dernes condi 
tions for merging using distance threshold Without resolving border details between 
textured regions, we use the homogeneous rectangular blocks of texture within each 
image to perform indexing m the database This approach in general places no 
limitations on how features are extracted from image blocks W'e ha^e used specific 
feature sets based on the QMF waielet decomposition because of the discrimination 
performance and the benefits that this lepresentation offers in a database application 
The quad tree method offers an efficient approach towards segmenting and repiesent 
mg textures present in the image and provides a general framework by which other 
discriminating features can be used for image segmentation Clustering method used 
here is a standard method for grouping data For better clustering linear transforma 
tion IS applied on the features \ectors Our experimental results, gi\en in chapter 5, 
show that we have obtained df sired results of retrieval Referring to the Table 1 and 
2 we conclude that the search results point to the segments of the database images 
that have similar texture information as the query image 
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6 1 Further Work 

As content based indexing has many applications for efficient retne\al of \isual in 
formation from multimedia databases further research can focus on dexelopment of 
content based indexing techniques which are domain independent and can be auto- 
mated 
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